5 research outputs found
Multi-Context Dual Hyper-Prior Neural Image Compression
Transform and entropy models are the two core components in deep image
compression neural networks. Most existing learning-based image compression
methods utilize convolutional-based transform, which lacks the ability to model
long-range dependencies, primarily due to the limited receptive field of the
convolution operation. To address this limitation, we propose a
Transformer-based nonlinear transform. This transform has the remarkable
ability to efficiently capture both local and global information from the input
image, leading to a more decorrelated latent representation. In addition, we
introduce a novel entropy model that incorporates two different hyperpriors to
model cross-channel and spatial dependencies of the latent representation. To
further improve the entropy model, we add a global context that leverages
distant relationships to predict the current latent more accurately. This
global context employs a causal attention mechanism to extract long-range
information in a content-dependent manner. Our experiments show that our
proposed framework performs better than the state-of-the-art methods in terms
of rate-distortion performance.Comment: Accepted to IEEE 22 International Conference on Machine Learning
and Applications 2023 (ICMLA) - Selected for Oral Presentatio
Frequency Disentangled Features in Neural Image Compression
The design of a neural image compression network is governed by how well the
entropy model matches the true distribution of the latent code. Apart from the
model capacity, this ability is indirectly under the effect of how close the
relaxed quantization is to the actual hard quantization. Optimizing the
parameters of a rate-distortion variational autoencoder (R-D VAE) is ruled by
this approximated quantization scheme. In this paper, we propose a
feature-level frequency disentanglement to help the relaxed scalar quantization
achieve lower bit rates by guiding the high entropy latent features to include
most of the low-frequency texture of the image. In addition, to strengthen the
de-correlating power of the transformer-based analysis/synthesis transform, an
augmented self-attention score calculation based on the Hadamard product is
utilized during both encoding and decoding. Channel-wise autoregressive entropy
modeling takes advantage of the proposed frequency separation as it inherently
directs high-informational low-frequency channels to the first chunks and
conditions the future chunks on it. The proposed network not only outperforms
hand-engineered codecs, but also neural network-based codecs built on
computation-heavy spatially autoregressive entropy models.Comment: Accepted to 30 IEEE International Conference on Image
Processing (ICIP 2023
Multi-spectral Entropy Constrained Neural Compression of Solar Imagery
Missions studying the dynamic behaviour of the Sun are defined to capture
multi-spectral images of the sun and transmit them to the ground station in a
daily basis. To make transmission efficient and feasible, image compression
systems need to be exploited. Recently successful end-to-end optimized neural
network-based image compression systems have shown great potential to be used
in an ad-hoc manner. In this work we have proposed a transformer-based
multi-spectral neural image compressor to efficiently capture redundancies both
intra/inter-wavelength. To unleash the locality of window-based self attention
mechanism, we propose an inter-window aggregated token multi head self
attention. Additionally to make the neural compressor autoencoder shift
invariant, a randomly shifted window attention mechanism is used which makes
the transformer blocks insensitive to translations in their input domain. We
demonstrate that the proposed approach not only outperforms the conventional
compression algorithms but also it is able to better decorrelates images along
the multiple wavelengths compared to single spectral compression.Comment: Accepted to IEEE 22 International Conference on Machine
Learning and Applications 2023 (ICMLA
Context-Aware Neural Video Compression on Solar Dynamics Observatory
NASA's Solar Dynamics Observatory (SDO) mission collects large data volumes
of the Sun's daily activity. Data compression is crucial for space missions to
reduce data storage and video bandwidth requirements by eliminating
redundancies in the data. In this paper, we present a novel neural
Transformer-based video compression approach specifically designed for the SDO
images. Our primary objective is to efficiently exploit the temporal and
spatial redundancies inherent in solar images to obtain a high compression
ratio. Our proposed architecture benefits from a novel Transformer block called
Fused Local-aware Window (FLaWin), which incorporates window-based
self-attention modules and an efficient fused local-aware feed-forward (FLaFF)
network. This architectural design allows us to simultaneously capture
short-range and long-range information while facilitating the extraction of
rich and diverse contextual representations. Moreover, this design choice
results in reduced computational complexity. Experimental results demonstrate
the significant contribution of the FLaWin Transformer block to the compression
performance, outperforming conventional hand-engineered video codecs such as
H.264 and H.265 in terms of rate-distortion trade-off.Comment: Accepted to IEEE 22 International Conference on Machine
Learning and Applications 2023 (ICMLA) - Selected for Oral Presentatio
Neural-based Compression Scheme for Solar Image Data
Studying the solar system and especially the Sun relies on the data gathered
daily from space missions. These missions are data-intensive and compressing
this data to make them efficiently transferable to the ground station is a
twofold decision to make. Stronger compression methods, by distorting the data,
can increase data throughput at the cost of accuracy which could affect
scientific analysis of the data. On the other hand, preserving subtle details
in the compressed data requires a high amount of data to be transferred,
reducing the desired gains from compression. In this work, we propose a neural
network-based lossy compression method to be used in NASA's data-intensive
imagery missions. We chose NASA's SDO mission which transmits 1.4 terabytes of
data each day as a proof of concept for the proposed algorithm. In this work,
we propose an adversarially trained neural network, equipped with local and
non-local attention modules to capture both the local and global structure of
the image resulting in a better trade-off in rate-distortion (RD) compared to
conventional hand-engineered codecs. The RD variational autoencoder used in
this work is jointly trained with a channel-dependent entropy model as a shared
prior between the analysis and synthesis transforms to make the entropy coding
of the latent code more effective. Our neural image compression algorithm
outperforms currently-in-use and state-of-the-art codecs such as JPEG and
JPEG-2000 in terms of the RD performance when compressing extreme-ultraviolet
(EUV) data. As a proof of concept for use of this algorithm in SDO data
analysis, we have performed coronal hole (CH) detection using our compressed
images, and generated consistent segmentations, even at a compression rate of
bits per pixel (compared to 8 bits per pixel on the original data)
using EUV data from SDO.Comment: Accepted for publication in IEEE Transactions on Aerospace and
Electronic Systems (TAES). arXiv admin note: text overlap with
arXiv:2210.0647